Neuro-Symbolic Approaches for Context-Aware Human Activity Recognition

Deep Learning models are a standard solution for sensor-based Human Activity Recognition (HAR), but their deployment is often limited by labeled data scarcity and models' opacity. Neuro-Symbolic AI (NeSy) provides an interesting research direction to mitigate these issues by infusing knowledge about context information into HAR deep learning classifiers. However, existing NeSy methods for context-aware HAR require computationally expensive symbolic reasoners during classification, making them less suitable for deployment on resource-constrained devices (e.g., mobile devices). Additionally, NeSy approaches for context-aware HAR have never been evaluated on in-the-wild datasets, and their generalization capabilities in real-world scenarios are questionable. In this work, we propose a novel approach based on a semantic loss function that infuses knowledge constraints in the HAR model during the training phase, avoiding symbolic reasoning during classification. Our results on scripted and in-the-wild datasets show the impact of different semantic loss functions in outperforming a purely data-driven model. We also compare our solution with existing NeSy methods and analyze each approach's strengths and weaknesses. Our semantic loss remains the only NeSy solution that can be deployed as a single DNN without the need for symbolic reasoning modules, reaching recognition rates close (and better in some cases) to existing approaches

SelfAct: Personalized Activity Recognition based on Self-Supervised and Active Learning

Supervised Deep Learning (DL) models are currently the leading approach for sensor-based Human Activity Recognition (HAR) on wearable and mobile devices. However, training them requires large amounts of labeled data whose collection is often time-consuming, expensive, and error-prone. At the same time, due to the intra- and inter-variability of activity execution, activity models should be personalized for each user. In this work, we propose SelfAct: a novel framework for HAR combining self-supervised and active learning to mitigate these problems. SelfAct leverages a large pool of unlabeled data collected from many users to pre-train through self-supervision a DL model, with the goal of learning a meaningful and efficient latent representation of sensor data. The resulting pre-trained model can be locally used by new users, which will fine-tune it thanks to a novel unsupervised active learning strategy. Our experiments on two publicly available HAR datasets demonstrate that SelfAct achieves results that are close to or even better than the ones of fully supervised approaches with a small number of active learning queries

Ultrasound Detection of Subquadricipital Recess Distension

Joint bleeding is a common condition for people with hemophilia and, if untreated, can result in hemophilic arthropathy. Ultrasound imaging has recently emerged as an effective tool to diagnose joint recess distension caused by joint bleeding. However, no computer-aided diagnosis tool exists to support the practitioner in the diagnosis process. This paper addresses the problem of automatically detecting the recess and assessing whether it is distended in knee ultrasound images collected in patients with hemophilia. After framing the problem, we propose two different approaches: the first one adopts a one-stage object detection algorithm, while the second one is a multi-task approach with a classification and a detection branch. The experimental evaluation, conducted with $483$ annotated images, shows that the solution based on object detection alone has a balanced accuracy score of $0.74$ with a mean IoU value of $0.66$, while the multi-task approach has a higher balanced accuracy value ($0.78$) at the cost of a slightly lower mean IoU value

Modeling and reasoning with ProbLog: an application in recognizing complex activities

Smart-home activity recognition is an enabling tool for a wide range of ambient assisted living applications. The recognition of ADLs usually relies on supervised learning or knowledge-based reasoning techniques. In order to overcome the well-known limitations of those two approaches and, at the same time, to combine their strengths to improve the recognition rate, many researchers investigated Markov Logic Networks (MLNs). However, MLNs require a non-trivial effort by experts to properly model probabilities in terms of weights. In this paper, we propose a novel method based on ProbLog. ProbLog is a probabilistic extension of Prolog, which allows to explicitly define probabilistic facts and rules. With respect to MLN, the inference mode of ProbLog is based on the closed-world assumption and it has faster response times. We propose a simple and flexible ProbLog model, which we exploit to recognize complex ADLs in an online fashion. Considering a dataset with 21 subjects, our results show that our method reaches high F-measure (83%). Moreover, we also show that the response time of ProbLog is satisfying for real-time applications

Analysis of long-term abnormal behaviors for early detection of cognitive decline

Several researchers have proposed methods and designed systems for the automatic recognition of activities and abnormal behaviors with the goal of early detecting cognitive impairment. In this paper, we propose LOTAR, a hybrid behavioral analysis system coupling state of the art machine learning techniques with knowledge-based and data mining methods. Medical models designed in collaboration with cognitive neuroscience researchers guide the recognition of short- and long-term abnormal behaviors. In particular, we focus on historical behavior analysis for long-term anomaly detection, which is the principal novelty with respect to our previous works. We present preliminary results obtained by evaluating the method on a dataset acquired during three months of experimentation in a real patient's home. Results indicate the potential utility of the system for long-term monitoring of cognitive health

Unsupervised recognition of interleaved activities of daily living through ontological and probabilistic reasoning

Recognition of activities of daily living (ADLs) is an enabling technology for several ubiquitous computing applications. In this field, most activity recognition systems rely on supervised learning methods to extract activity models from labeled datasets. An inherent problem of that approach consists in the acquisition of comprehensive activity datasets, which is expensive and may violate individuals' privacy. The problem is particularly challenging when focusing on complex ADLs, which are characterized by large intra- and inter-personal variability of execution. In this paper, we propose an unsupervised method to recognize complex ADLs exploiting the semantics of activities, context data, and sensing devices. Through ontological reasoning, we derive semantic correlations among activities and sensor events. By matching observed sensor events with semantic correlations, a statistical reasoner formulates initial hypotheses about the occurred activities. Those hypotheses are refined through probabilistic reasoning, exploiting semantic constraints derived from the ontology. Extensive experiments with real-world datasets show that the accuracy of our unsupervised method is comparable to the one of state of the art supervised approaches